\section{Conclusion}\label{sec:con}

We have built a non-intrusive, effective, distributed monitoring tool to facilitate the Hadoop program debugging process. The tool correlates rich MapReduce metrics extracted from Hadoop logs and per-job/per-task level operating system metrics.
We have proposed a multi-view visualization scheme to effectively present these metrics, as well as a view-zooming model to help programmers better reason about job execution progress and system behavior. The preliminary results are promising: our tool can successfully diagnose several performance problems that previous monitoring tools cannot.

As for future work, we plan to instrument the Hadoop framework in the hope of exploring more information from Hadoop metrics APIs.
Another potential direction is to achieve online automatic data analysis based on the aggregated metrics. 
Instead of storing temporary data on RRD database, we also would like to find an effective way to maintain long-term storage of the collected metrics.
